This invention relates to solid state image sensors and associated electronics, and more particularly, to solid state image sensors which are configured to be of a minimum size and used within miniature computer systems known as palm top computers, personal digital assistants (PDA), or hand-held computers/organizers.
The three most common solid state image sensors include charged coupled devices (CCD), charge injection devices (CID) and photo diode arrays. In the mid-1980s, complementary metal oxide semiconductors (CMOS) were developed for industrial use. CMOS imaging devices offer improved functionality and simplified system interfacing. Furthermore, many CMOS imagers can be manufactured at a fraction of the cost of other solid state imaging technologies.
The CCD device is still the preferred type of imager used in scientific applications. Only recently have CMOS-type devices been improved such that the quality of imaging compares to that of CCD devices. However, there are enormous drawbacks with CCD devices. Two major drawbacks are that CCD device have immense power requirements, and the amount of processing circuitry required for a CCD imager always requires the use of a remote processing circuitry module which can process the image signal produced by the CCD imager. Also, because of the type of chip architecture used with CCD devices, on-chip processing is impossible. Therefore, even timing and control circuitry must be remoted from the CCD imager plane. Therefore, CCD technology is the antithesis of xe2x80x9ccamera on a chipxe2x80x9d technology discussed below.
One particular advance in CMOS technology has been in the active pixel-type CMOS imagers which consist of randomly accessible pixels with an amplifier at each pixel site. One advantage of active pixel-type imagers is that the amplifier placement results in lower noise levels. Another major advantage is that these CMOS imagers can be mass produced on standard semiconductor production lines. One particularly notable advance in the area of CMOS imagers including active pixel-type arrays is the CMOS imager described in U.S. Pat. No. 5,471,515 to Fossum, et al. This CMOS imager can incorporate a number of other different electronic controls that are usually found on multiple circuit boards of much larger size. For example, timing circuits, and special functions such as zoom and anti-jitter controls can be placed on the same circuit board containing the CMOS pixel array without significantly increasing the overall size of the host circuit board. Furthermore, this particular CMOS imager requires 100 times less power than a CCD-type imager. In short, the CMOS imager disclosed in Fossum, et al. has enabled the development of a xe2x80x9ccamera on a chip.xe2x80x9d
Passive pixel-type CMOS imagers have also been improved so that they too can be used in an imaging device which qualifies as a xe2x80x9ccamera on a chip.xe2x80x9d In short, the major difference between passive and active CMOS pixel arrays is that a passive pixel-type imager does not perform signal amplification at each pixel site. One example of a manufacturer which has developed a passive pixel array with performance nearly equal to known active pixel devices and compatible with the read out circuitry disclosed in the U.S. Pat. No. 5,471,515 is VLSI Vision, Ltd., 1190 Saratoga Avenue, Suite 180, San Jose, Calif. 95129. A further description of this passive pixel device may be found in co-pending application, Ser. No. 08/976,976, entitled xe2x80x9cReduced Area Imaging Devices Incorporated Within Surgical Instruments,xe2x80x9d now U.S. Pat. No. 5,986,693, and is hereby incorporated by reference.
In addition to the active pixel-type CMOS imager which is disclosed in U.S. Pat. No. 5,471,515, there have been developments in the industry for other solid state imagers which have resulted in the ability to have a xe2x80x9ccamera on a chip.xe2x80x9d For example, Suni Microsystems, Inc. of Mountain View, Calif., has developed a CCD/CMOS hybrid which combines the high quality image processing of CCDs with standard CMOS circuitry construction. In short, Suni Microsystems, Inc. has modified the standard CMOS and CCD manufacturing processes to create a hybrid process providing CCD components with their own substrate which is separate from the P well and N well substrates used by the CMOS components. Accordingly, the CCD and CMOS components of the hybrid may reside on different regions of the same chip or wafer. Additionally, this hybrid is able to run on a low power source (5 volts) which is normally not possible on standard CCD imagers which require 10 to 30 volt power supplies. A brief explanation of this CCD/CMOS hybrid can be found in the article entitled xe2x80x9cStartup Suni Bets on Integrated Processxe2x80x9d found in Electronic News, Jan. 20, 1997 issue. This reference is hereby incorporated by reference for purposes of explaining this particular type of imaging processor.
Another example of a recent development in solid state imaging is the development of a CMOS imaging sensor which is able to achieve analog to digital conversion on each of the pixels within the pixel array. This type of improved CMOS imager includes transistors at every pixel to provide digital instead of analog output that enable the delivery of decoders and sense amplifiers much like standard memory chips. With this new technology, it may, therefore, be possible to manufacture a true digital xe2x80x9ccamera on a chip.xe2x80x9d This CMOS imager has been developed by a Stanford University joint project and is headed by Professor Abbas el-Gamal.
A second approach to creating a CMOS-based digital imaging device includes the use of an over-sample converter at each pixel with a one bit comparator placed at the edge of the pixel array instead of performing all of the analog to digital functions on the pixel. This new design technology has been called MOSAD (multiplexed over sample analog to digital) conversion. The result of this new process is low power usage, along with the capability to achieve enhanced dynamic range, possibly up to 20 bits. This process has been developed by Amain Electronics of Simi Valley, Calif. A brief description of both of the processes developed by Stanford University and Amain Electronics can be found in an article entitled xe2x80x9cA/D Conversion Revolution for CMOS Sensor?,xe2x80x9d September 1998 issue of Advanced Imaging. This article is also hereby incorporated by reference for purposes of explaining these particular types of imaging processors.
Yet another example of a recent development with respect to solid state imaging is an imaging device developed by ShellCase, of Jerusalem, Israel. In an article entitled xe2x80x9cA CSP Optoelectronic Package for Imaging and Light Detection Applicationsxe2x80x9d (A. Badihi), ShellCase introduces a diesized, ultrathin optoelectronic package which is completely packaged at the wafer level using semiconductor processing. In short, ShellCase provides a chip scale package (CSP) process for accepting digital image sensors which may be used, for example, in miniature cameras. The die-sized, ultrathin package is produced through a wafer level process which utilizes optically clear materials and completely encases the imager die. This packaging method, ideally suited for optoelectronic devices, results in superior optical performance and form factor not available by traditional image sensors. This article is also incorporated by reference for purposes of explaining ShellCase""s chip scale package process.
Yet another example of a recent development with respect to solid state imaging is shown in U.S. Pat. No. 6,020,581 entitled xe2x80x9cSolid State CMOS Imager Using Silicon on Insulator or Bulk Silicon.xe2x80x9d This patent discloses an image sensor incorporating a plurality of detector cells arranged in an array wherein each detector cell as a MOSFET with a floating body and operable as a lateral bipolar transistor to amplify charge collected by the floating body. This invention overcomes problems of insufficient charge being collected in detector cells formed on silicon on insulator (SOI) substrates due to silicon thickness and will also work in bulk silicon embodiments.
The above-mentioned developments in solid state imaging technology have shown that xe2x80x9ccamera on a chipxe2x80x9d devices will continue to be enhanced not only in terms of the quality of imaging which may be achieved, but also in the specific construction of the devices which may be manufactured by new breakthrough processes.
Although the xe2x80x9ccamera on a chipxe2x80x9d concept is one which has great merit for application in many industrial areas, a need still exists for a reduced area imaging device which can be used in even the smallest type of industrial application. Recently, devices known as palm top computers, PDA(s), or hand-held computers have become very popular items. Essentially, these PDAs are miniature computers, small enough to be held in the hand, which have various software programs available to a user including word processing, e-mail, and organization software for addresses/phone books, etc.
One example of a U.S. patent disclosing a type of a PDA includes U.S. Pat. No. 5,900,875. This patent is incorporated herein by reference for purposes of illustrating an example of a PDA including basic functionality for such a device. In a recent article entitled xe2x80x9cPalm, Inc. Gets Ready for New Handsxe2x80x9d appearing in the Wallstreet Journal, a number of soon to be commercially available PDAs are disclosed. One such device disclosed in this article is known as the xe2x80x9cHand Spring Visor Deluxe.xe2x80x9d This device will soon be available which allows a user to accommodate pagers, MP3 players, still digital cameras and other devices.
It is one general object of this invention to provide a video system in combination with a standard PDA enabling a user to take video images by a very small camera module incorporated within the PDA, view the video images taken on a video viewscreen incorporated within the PDA, and to have the capability to store, download the video images, and send the video images electronically through a communications network.
Another object of this invention is to provide a PDA with the ability to not only transmit video images taken by the camera module, but also to receive video images sent from a remote location via the communications network, and to view such received video images on the video view screen of the PDA. Accordingly, the invention is ideally suited for video teleconferencing.
It is another object of this invention to provide a reduced area imaging device incorporated within a PDA which takes advantage of xe2x80x9ccamera on a chipxe2x80x9d technology, but to rearrange the video processing circuitry in a selective stacked relationship so that the camera module has a minimum profile.
It is yet another object of this invention to provide imaging capability for a PDA wherein the video camera used is of such small size that it can be stored in the PDA when not in use. The camera module is attached to the PDA by a retractable cord which enables the imaging device to be used to image anything at which the camera module is pointed by the user without having to also move the PDA away from the view of the user.
In all applications, to include use of the imaging device of this invention with a PDA, xe2x80x9ccamera on a chipxe2x80x9d technology can be improved with respect to reducing its profile area, and incorporating such a reduced area imaging device within a PDA such that minimal size and weight is added to the PDA, and further that the imaging device can be used to image selected targets by the user.
In accordance with the present invention, reduced area imaging devices are provided in combination with a hand-held computer or PDA. The term xe2x80x9cimaging devicexe2x80x9d as used herein describes the imaging elements and processing circuitry which is used to produce a video signal which may be accepted by both a standard video device such as a television or video monitor accompanying a personal computer, and a small LCD screen which is incorporated within the PDA. The term xe2x80x9cimage sensorxe2x80x9d as used herein describes the components of a solid state imaging device which captures images and stores them within the structure of each of the pixels in the array of pixels found in the imaging device. As further discussed below, the timing and control circuits can be placed either on the same planar structure as the pixel array, in which case the image sensor can also be defined as an integrated circuit, or the timing and control circuitry can be placed remote from the pixel array. The terms xe2x80x9cvideo signalxe2x80x9d or xe2x80x9cimage signalxe2x80x9d as used herein, and unless otherwise more specifically defined, refer to an image which at some point during its processing by the imaging device, is found in the form of electrons which have been placed in a specific format or domain. The term xe2x80x9cprocessing circuitryxe2x80x9d as used herein refers to the electronic components within the imaging device which receive the image signal from the image sensor and ultimately place the image signal in a usable format. The terms xe2x80x9ctiming and control circuitsxe2x80x9d or xe2x80x9ctiming and control circuitryxe2x80x9d as used herein refer to the electronic components which control the release of the image signal from the pixel array.
In a first arrangement, the image sensor, with or without the timing and control circuitry, may be placed at the distal tip of a very small camera module which is attached by a cable or cord to the PDA, while the remaining processing circuitry may be placed within the housing of the PDA.
In a second arrangement, the image sensor and the processing circuitry may all be placed in a stacked arrangement of miniature circuit boards or planar circuit structures and positioned at the distal tip of the camera module. In this second arrangement, the pixel array of the image sensor may be placed by itself on its own circuit board while the timing and control circuitry and processing circuitry are placed on one or more other circuit boards, or the circuitry for timing and control may be placed with the pixel array on one circuit board, while the remaining processing circuitry can be placed on one or more of the other circuit boards.
In yet another alternative arrangement, the pixel array, timing and control circuits, and some of the processing circuitry can be placed near the distal end of the camera module on two or more circuit boards with the remaining part of the processing circuitry being placed in the housing of the PDA.
For the arrangement or configuration of the imaging device which calls for the array of pixels and the timing and control circuitry to be placed on the same circuit board, only one conductor is required in order to transmit the image signal to the video processing circuitry. When the timing and control circuits are incorporated onto other circuit boards, a plurality of connections are required in order to connect the timing and control circuitry to the pixel array, and then the one conductor is also required to transmit the image signal back to the video processing circuitry.
As mentioned above, the invention disclosed herein can be considered an improvement to a PDA wherein the improvement comprises a video system. The video system would include the video view screen or monitor attached to the PDA, the camera module, as well as supporting video processing circuitry for the imaging device. In yet another aspect, the invention disclosed herein can also be considered an improvement to a PDA wherein the improvement comprises a novel imaging device, preferably of CMOS construction. For this improvement comprising the imaging device, the imaging device includes the array of pixels, and the supporting video processing circuitry for providing a video ready signal.
This video ready signal may be formatted by the video processing circuitry for viewing on a NTSC/PAL compatible device such as television, or for viewing on a VGA compatible device such as a monitor of a personal computer. Of course, the video ready signal is formatted for viewing the video images on the video view screen incorporated within the PDA.
In yet another aspect, the invention disclosed herein can also be considered an improvement to a PDA wherein the improvement comprises a combination of a video system, and wireless telephone communication means for transmitting and receiving both audio and video signals. In this aspect, the invention has functionality for transmitting and receiving audio and video signals via the communications network. One example of a U.S. patent disclosing wireless remote communications between a personal computer and a PDA or miniature hand held computer is U.S. Pat. No. 6,034,621. This patent is hereby incorporated by reference in its entirety for purposes of disclosing means by which data can be exchanged between the hand held computer and a personal computer, to include video and audio signals. The specific example in this patent which readily lends itself to the communication network incorporated within this invention is found at FIG. 4 of this ""621 patent. The discussion further below outlines this particular communication network.
In yet another aspect, the invention disclosed herein can also be considered an improvement to a PDA wherein the improvement comprises a video system, and a standard wireless telephone communication means for transmitting and receiving audio signals. In this aspect, the PDA simply includes a standard wireless/cellular phone connected externally on the PDA which enables the user to conduct well-known wireless/telephone communications. This wireless/cellular communication means can be in addition to the wireless telephone communication means for transmitting and receiving both audio and video signals discussed immediately above with respect to the U.S. Pat. No. 6,034,621.
Another example of a U.S. patent disclosing basic mobile phone technology including a discussion of basic phone circuitry is U.S. Pat. No. 6,018,670. This patent is hereby incorporated by reference in its entirety for purposes of disclosing standard or basic mobile phone technology and supporting circuitry.
Accordingly, the invention disclosed herein has utility with respect to an overall combination of elements, as well as various sub-combination of elements.